Adiabatic feedback control of Hamiltonian systems. 
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We study feedback control of classical Hamiltonian systems with the controlling parameter varying 
slowly in time. The control aims to change system's energy. We show that the control problems can 
be solved with help of an adiabatic invariant that generalizes the conservation of the phase-space 
volume to control situations. New mechanisms of control for achieving heating, cooling, entropy 
reduction and particle trapping are found. The feedback control of a many-body system via one of 
its coordinates is discussed. The results are illustrated by two basic models of non-linear physics. 
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Physics met control theory yet in XlX'th century: the 
notorious Maxwell's demon poses a problem of building a 
control system that will reduce the entropy (increase the 
order) of a statistical system 0,12 • Founders of cybernet- 
ics recognized the entropy reduction as one of the basic 
goals of control [^Q. This is especially relevant, since 
the statistical description — in particular, thermodynam- 
ical relations and concepts such as entropy — are needed 
not only for macroscopic systems, but also for few-body 
chaotic ones @. Deep implications of the entropy- 
information-control relations were studied in The- 
oretical and experimental methods of controlling physical 
systems have recently undergone an explosive develop- 
ment 0, B S 0^ • The focus is on non-linear and chaotic 
systems in view of their numerous applications. 

The potential of physical ideas applicable to the con- 
trol science is, however, far from being exhausted. Here 
we explore a feedback control of ergodic Hamiltonian sys- 
tems, where a slow motion of the controlling parameter 
leads to the existence of an adiabatic invariant. This 
fact helps to solve the control problem on a general level. 
Feedback means that the control parameter depends on 
the current state of the system, while in the non- feedback 
control the motion of the control parameter is prescribed. 
In this respect we built up on the non-feedback adiabatic 
control of ergodic systems, where the adiabatic invari- 
ance of the phase-space volume is well-known 0, . 
Despite of its general importance, from the control view- 
point this method has several drawbacks: i) The mi- 
crocanonic entropy is conserved (or increases for non- 
adiabatic processes ^3); ^o that the goal of the control 
is not achieved, ii) The method docs not function for 
the important case of cyclic influences of the controlling 
parameters, i.e., it insists on permanent modifications of 
the system. We show that these drawbacks are absent 
for adiabatic feedback control, while the main advan- 
tage — independence on the details of the system under 
assuming ergodicity of certain observables — is kept. 

Consider a system with N degrees of freedom and 
Hamiltonian H(j}, q, R). The equations of motion read 



where q = (gi, (/at) and p — {pi, ...,pn) are, respec- 
tively, the coordinates and momenta, and where R is the 
control parameter. The purpose of varying R in time is 
to change system's energy E 'ma. desired way. This setup 
of control 0,13 especially relevant for chaotic^stems, 
since energy is the basic constant of motion [a, 1^1 • 

The change of R can be sizable, but the speed of R is 
small and depends on the dynamical variables p and q 
via a phase-space observable F (feedback). Behind the 
scene of this description is a measurement of F (sensor) 
and subsequent engineering (actuator) that leads to 



trR^F{z,R), 



(2) 



where tr is much larger than the characteristic time ts of 
the system (defined with respect to the R =const dynam- 
ics) . The derivative of the control parameter depending 
on the state of the system is standard in control practice 
00|. For a general F the problem is hardl2_tractable an- 
alytically. However, an important paper [iSj shows that 
an one-dimensional (N = 1) Hamiltonian system admits 
an adiabatic invariant provided 



F(z,i?) = /(i?)^(z). 



(3) 



Here ip and / depend on the fast and slow variables, re- 
spectively; these dependencies are thus factorized in JSJ. 
Assuming we extend this invariant to systems with 
ergodic observables and apply it to control problems. 

The below derivation is based on the fact that R and 
energy E are slow variables, since they change on times 
~ tr. For times ^ r ^ rs we have from (PEJ ISJ: 

dE 1 

— = -[Hizt+r,Rt+r)-Hizt,Rt)] (4) 
dr r 



t + T 



ds dH 



{Zs,Rs) 



ds ■ dH. ^ , 
— R .S ^ „ ( z g , R s ) , 
T dR^ ' 



p^ -dgH{p,q,R), q = dpH{p,q,R), 



(1) 



1 /■ "'"^ ds dH T 

^-.f{Rt) -v{z)jr^{zs.Rt) + o{-). (5) 

tr Jt T dR tr 

The last integral refers to the dynamics with Rt =:const. 
Now note the Liouville theorem dz — dzt and energy 
conservation H{zt+T) = H{zt) = Ft, and denote w{z) = 



2 



(p{z) diiH{z, Rt). Noting the microcanonical distribution 

^^-) = ' W,';^f ' ^iE)-I^^O[E-Hiz,R)], 

where S{x) and 0{x) are, respectively, the delta and step 
function, and where is the phase-space volume, we get 



1 



t + T 



dzw{z)M{z,Et) ^ - J ds J dzw{z)M{z, Et){6) 
= J dztM{zuEt)^ dsw{z[zt,s]),{7) 



where z[zt,s] is the trajectory at time s with the initial 
condition Zf. If w{z^R) is an ergodic observable of the 
Rt =const dynamics, then for r 3> ts the time-average 
in |2J does not depend on the initial condition zt , the 
integration over zt in Q drops out, and we get from ® 
that the time-average in 10) is equal to the microcanoni- 
cal average at the energy Et: 

dE^fiRtl (8) 
dr TR J dR 



Assuming ergodicity oi (p{z) we get from 

^^—f{Rt)[dzM{z,EuRtMz). (9) 
dr tr J 

It is seen that (jS||5)l amount to the conservation condition 

^=0, I(E,R)= [ dzip(z)e[E- H(z,R)]. (10) 
dr J 

I does not depend on the initial condition z{ti) of the 
trajectory. Given the initial values Ei and Ri, the final 
E{ and Rf are found self- consistently from (jSJ and from 



I{Ei,Ri) = IiEf,Rf). 



(11) 



For (p = 1 (no feedback) we get from (|ll)|l the conserva- 
tion of the phase-space volume il. For an isolated ergodic 
system the entropy is defined as S" = In 6] . This defini- 
tion satisfies to all reasonable features of entropy, e.g., for 
the temperature defined via 1/T = dE^'n.^iE), the inte- 
gration by parts leads to equipartition 0: (xdxH) = T, 
where a; is any canonical coordinate of momentum, and 
(...) is the average over microcanonical distribution l^. 
For non-adiabatic processes S normally increases, thus 
confirming the second law ^| . In the standard ther- 
modynamical limit S goes to the more usual expression 
S = hi[dE^] ifij- Neither of the above important features 
holds if we apply S for a finite system . 

For the feedback case (p I, (|ll|l shows that is not 
conserved, and the entropy S can decrease, as we see 
below. Thus the basic goal of control will be attained. 

As an application consider n particles moving between 
an infinite wall located at g = and a piston with the 



controlled coordinate q = R > 0. The Hamiltonian is 

2 

H = X]r=i ('^i masses) apart from the infinite 
wall interactions that lead to elastic scattering and apart 
from weak inter-particle couplings that ensure the ergod- 
icity of the system Q. The simplest type of feedback 
is realized via the coordinate q of one of the particles: 
(fi — fiq). The conservation of / leads from 



E 



>/2 



dqifiq) q''-^=E\ 



1/2 



Rt 



dqipiq)q^-\ (12) 



We start with n = 1 (Fermi accelerator), where the 
R =const motion is obviously ergodic. This model was 
intensively studied in plasma and accelerator physics, 
nonlinear physics and astrophysics 0, Il5l |. A question 
that created the interest to the model is how to increase 
system's energy E (heating)? The non-feedback adia- 
batic control is able to heat up the system only upon 
decreasing R, since the phase-space volume oc \fER is 
conserved. This is not realistic, since involves a perma- 
nent and substantial modification of system's parameter. 
A non-adiabatic heating studied in llSj amounts to an 
oscillating motion of the piston, e.g., R{t) = Ri+aTsva ^ 
with period T. The maximal achievable energy is E'max ~ 
Once Ri is fixed, -Emax grows by increas- 
ing a (large amplitude) or by decreasing T (large piston 
velocity). Thus it is impossible to increase the energy 
as much as desired 0|. The adiabatic feedback control 
solves this problem as follows. Take f{q) = q — a, where 
a > is a constant. We get from H12|l 



(13) 



'EiRi{Ri - 2a) = ^/EiRi{Ri - 2a). 



Taking R{ — > 2a (provided Ri is not close to 2a), the final 
energy is increased as much as desired. Eq. (jnj for the 
piston is Tp-^R 



^{R — 2a), or (taking / =const) 



R{t) = 2a+ {Ri - 2a)e 



fr/{2Ta) 



(14) 



For / < 0, |/r| 3> tr suffices to make R[ very close to 
2a. Eq. H12|l also shows that for a more general feedbacks 
function if, the strong heating exists for dq(p{q) 0. 

There are three regimes of (|T^ considered separately: 
i) for i?i -I- i?f > 2a > i?f > Ri the heating is achieved 
by expanding the system. Taking the feedback out and 
squeezing R adiabatically back to Ri, we heat up the 
particle even more and complete the cycle. The reason 
of the feedback heating is seen from ((21l31l: R = \ f\{ci — 
q). When R > a, the piston tends to move towards the 
approaching particle, and during the resulting collision 
the particle gains energy. However, R > when the 
particle is far from the piston, and thus in average R 
increases, ii) For Ri > R[ > 2a the particle is heated by 
squeezing. Hi) The case Rf < Ri < 2a and Ri -\- Rf < 2a 
is interesting, since in contrast to the above two cases the 
heating is not strong, but leads to the entropy decrease. 
Now in ((n|l we should take / > 0. 
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The abiUty of energy decreasing (coohng) is not less 
important in applications [loj . A scenario of cooling is 
illustrated by in the regime Ri + R{ > 2a > Ri > Rf, 
which is realized by taking / > in (|14|l . Now R = 
f{q — a), and moving out of the approaching particle, 
the piston gains energy. In the second step we take the 
feedback out and expand the piston back to its original 
position Ri, thereby cooling the system even more, reduc- 
ing its entropy and completing the cycle. To lower the 
energy as much as desired we need several cycles. This 
contrasts the heating situation, where one cycle sufficed. 
This is because dq ip{q) can turn to zero, but cannot 
be infinite, since gives the speed of R. Note that with- 
out feedback there is no obvious cooling cyclic process. 

For a finite number of particles n one can design from 
(I12|l cooling and heating processes analogously to the n — 
1 case. Extensions to more general models of chaotic 
systems 0|, e.g., bilhards ^ are straightforward. 

What about controlling a many-body ergodic system, 
where n ^ 1 and the feedback still goes over one co- 
ordinate: ip = ip{q)f This coordinate may belong to 
the particle with a large mass that moves slowly (due 
to equipartition) and is amenable for the measurement. 
Now note that (p{q) ^ e''"* (|6| ^ 1) is unphysical: if 
& > this will lead to a huge speed of L on the short 
times; see (|2 Ell- If < this speed will be too small 
and the control takes a huge time. Then for n ^ 1 and 
R > 1 the integral in (|12|l is dominated by the right end: 
/ ~ ip{R)E^/'^R^. This brings for the change of the mi- 
crocanonical entropy S — nhi[R-\fE]: Si — Si = In . 
Normally S'f — 5i ^ 1, though with a precise tuning we 
can achieve Sf — Si ~ Inn; take ^p{R) — R — a and 
R{ — a . Thus when controlling a macroscopic sys- 

tem via a single degree of freedom, the effect of feedback 
is small. This conclusion is more general: if the feedback 
uses the coordinate q, ifTTijl impHes I = J dq(p{q)e^^'^\ 
where S{q) = \nU,{q) is the entropy of the system with 
a fixed value of q. Once normally S{q) ^ n (for n':^ 1), 
the conclusion follows along the above line. 

Our last example is interesting in two respects. First 
this is a paradigmatic model of non-linear science 5j 
and control theory 0: the oscillator iJ = ^ — Rcosq, 
where the amplitude i? > is taken as the control vari- 
able. The periodic boundary conditions are ensured by 
— TT < q < TT and ip{q,p) = (p{q + 2iT,p). The model is 
basic for particle-wave interactions, where the potential 
—Rcosq models a harmonic wave (in the system of ref- 
erence where the wave is standing), and where E > R 
and E < R refer to the free (rotating) and captured 
(oscillating) particles, respectively Second, 
the system is non-ergodic: ior E > R the phase-space 
of the R =const dynamics consists of two ergodic com- 
ponents supporting the rotational motion with, respec- 
tively, p > and p < (the rotating particle never 
changes the sign of its momentum, while the energy is 



insensitive to this sign). In contrast, the oscillatory mo- 
tion for i? > > —R is globally ergodic. When R 
is time-dependent some trajectories cross the separatrix 
E = R and move from one ergodic component to an- 
other. Although the model is non-ergodic, / defined in 
(|10|l (with the integration over the whole phase-space ir- 
respectively of ergodic components) is conserved if the 
observable w{z) = tp{q,p)dfiH{q,p) = —ip{q,p)cosq is 
p — > —p symmetric. Indeed, if two ergodic components 
are possible for a given energy, w(z) does not depend 
on which component the paticle moves. The adiabatic 
condition 3> T5 can be satisfied as well: though the 
R =const motion on the separatrix has an infinite period 
(due to unstable fixed points q = ±7r), the fraction of 
particles trapped by the separatrix is negligible (measure 
zero) 0, so that is finite. Then the derivation (inHHl) 
applies. For non- feedback processes the conservation of 
the phase-space volume was first shown in 0| . 
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FIG. 1: The adiabatic invariant I and the phase-space volume 
Q, versus energy E. Dotted lines R = 2. Normal lines R — 1. 



We now focus on the following question: how to trap 
particles by means of a cyclic change of the amplitude 
R7 This is a version of the cooling problem. First note 
that the phase-space volume fl is expressed from as 
(changing variables for (|15|l as esinf — sin |) 



n = V2R[£{e^) - [1 - e^))C{e^)' 
n = V2R e f (e-2), 



where e = a/-^^, and where £{x) = JJ^ d^(l — 



< e < 1, (15) 

1 < e, (16) 

Tr/2 



X sin^ ^)i/2 and /C(x) = J^' d^ (1 - a; sin^ ^)-i/2 are the 
elliptic integrals. The two regimes (|15|l and H16|l corre- 
spond, respectively, to the trapped and free motions. 
As the feedback function we take ip — |cos(|)| = 



1 — sin^(|), which is invariant with respect to p — > —p 
and q ^ q + 27r. The adiabatic invariant reads from H1U|) 




e arcsin(- 



,1- 



0<e<l, (17) 



1 < e. (18) 



The two regimes have the same meaning as above. The 
behavior of / and fl is presented in Fig.^ Particles with 
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the energy above the initial separatrix E = Ri, but be- 
low the final separatrix E = R{ get trapped. In the non- 
feedback case the energy E responds to the amplitude 
increase in a non-linear way: the low energy motion de- 
creases its energy, as intuitively expected for almost har- 
monic oscillations, while the energies around the separa- 
trix increase. In contrast, the feedback linearizes the en- 
ergy response: now all energies decrease. This is because 
due to i? c!c I cos(|)|, the amplitude does not move when 
the particle is around the unstable fixed points q = ±7r, 
which are responsible for the strong nonlinearity. 

We shall now combine the two scenarios (with and 
without feedback) so as to get trapping via a cyclic 
change of R. Apply first the feedback control changing 
R from i?i = 1 to i?t = 2; see Fig. ^ Then apply the 
non-feedback control bringing R back to its initial value 
1. It is seen from Fig. ^ and (|15II18|1 that all energies 
E < Em = 1.249 get trapped after the cyclic process. 
Here Em is defined via solving two coupled equations: 
I{Em,Ri) = I{E*,Rf) and n{E = R^ Ri) = n{E*,Rf), 
i.e., in the feedback part of the process E^ E* — 
1.129, while in the non- feedback part E* goes precisely 
to the separatrix E = R^. These arguments are qualita- 

TABLE I; Numerical illustration of the cyclic trapping 
method. The energies are obtained from solving numeri- 
cally the oscillator equation of motion q + Rt sin q = G and 
the feedback equation TaRt = |cos|| in the time-interval 
from t = till ti = 2 x 10^. Here Ri = R{0) = 1 and 
tr — 10"'^. Then a non-feedback process was applied via 
equation of motion ij + [R{ti) — i^^Jsinq = till the final 
time ti = ti + TR[R{ti) - Ri], so that R{t[) = i?i = 1. The 
adiabatic invariant II18II is conserved: ~ 2 x 10"^. Ex- 

pectedly, its dependence on the initial conditions {q{0),p{0)) 
for a given -E(O) is very weak. The initial energies Ei — E{0) 
are above the separatrix E — 1 (free motion), while the final 
ones are below it (trapping). 



Ei 


1.26 


1.22 


1.2 


1.15 


1.1 


1. 


Ef 


0.9976 


0.9791 


0.9709 


0.9423 


0.9111 


0.8429 



five, since in the feedback case i?f is not given a priori, 
but comes out from the common solution of ^ \\\\ and 
can depend on Ei. However, the numerical results pre- 
sented and explained in Table I confirm the arguments. 

In conclusion, we presented an adiabatic feedback 
method for controlling classical Hamiltonian systems. 
The method is based on an adiabatic invariant. This 
requires ergodicity (time average is equal to the micro- 
canonical one) of certain observables and makes explicit 
the change of the microcanonical entropy due to control. 
We illustrated the method via two basic models of the 
non-linear science: Fermi's accelerator and the non-linear 
oscillator. For the Fermi accelerator — which was inten- 
sively studied as model of plasma heating 0, 0| — the 
adiabatic feedback method offers efficient schemes of 
heating that, in particular, may lead to entropy decrease. 



For the non-linear oscillator (wave-particle interaction) 
transitions are possible between ergodic and non-ergodic 
regimes of motion j^. They correspond to the particle 
(de)trapping by the wave. We designed schemes for par- 
ticle trapping via cyclic change of the wave amplitude, 
and noted how the feeedback changes qualitatively the 
response of this non-linear model. It was seen that when 
controlling an ergodic n-particle system (n ^ 1) via mon- 
itoring few of its coordinates, the entropy decrease A5 
due to feedback is AS* < Inn, while the entropy itself is 
S ^ n. This is not unexpected, since, e.g., the entropy 
difference between an organism and a chemical substance 
of the same weight is also much smaller than each of those 
entropies If, however, there is an efficiently con- 

trolled (AS ~ S) macroscopic system, we foresee it to be 
non-ergodic, so that the adiabatic invariance is absent. 
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